Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning

In this paper we revisit the idea of pseudo-labeling in context semi-supervised learning where a algorithm has access to small set labeled samples and large unlabeled samples. Pseudo-labeling works by applying pseudo-labels using model trained on combination any previously pseudo-labeled samples, iteratively repeating process self-training cycle. Current methods seem have abandoned approach favor consistency regularization that train models under different styles self-supervised losses standard supervised We empirically demonstrate can fact be competitive with state-of-the-art, while being more resilient out-of-distribution set. identify two key factors allow achieve such remarkable results (1) curriculum principles (2) avoiding concept drift restarting parameters before each obtain 94.91% accuracy CIFAR-10 only 4,000 68.87% top-1 Imagenet-ILSVRC 10%